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ABSTRACT 

A survey of computer scientists was conducted to 
identify factors that affect software complexity. A total of 160 
items were selected from the literature to include in a questionnaire 
sent to 425 individuals who were employees of computer-related 
businesses in Lawrence and Kansas City. The items were grouped into 
nine categories called system planning, system characteristics, 
system design, system testing, system documentation, system 
correctness, system clarity, programming style, and system 
management. Factor analysis and central tendency measures were used 
to analyze the data. Based on 147 respondents, 98% of the items were 
found to affect syst^^m complexity. Large item variance was attributed 
to the lack of formal education or experience of respondents in some 
areas of software development. The factors extracted in this study 
were found to be useful in regrouping items to determine the 
complexity of system attributes, system components, or of various 
phases of system development. The need for a weighting scheme to add 
component complexity to determine the overall system complexity was 
identified. It is suggested that replication of this study on a 
larger scale would prove useful and that the complexity of various 
system components be weighted to determine the overall complexity of 
the system. A list of references, a copy of the questionnaire, and 
factor matrices are also provided. (JB) 
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ABSTRACT 



A survey was conducted on computer scientists to 
Identify factors that affect software complexity. A total 
of 160 items were selected from the literature to include 
in the questionnaire. The items were grouped into nine 
categories called system planning, system characteristics, 
system design, system testing, system documentation, 
system correctness, system clarity, programming style, and 
system management. Factor analysis and central tendency 
measures were used to analyse data. 

Based on the data on 147 respondents, 98% of the 
items were found to affect system complexity. Large item 
variance was attributed to the lack of formal education or 
experience of respondents in some areas of software 
development. The categories used in the present study, 
were not found mutually exclusive. The factors extracted 
in this study were found useful in regrouping items to 
determine complexity of system attributes, system 
components, or of various phases of system development. 
The need for a weighting scheme to add component 
complexity to determine the overall system complexity was 
identified. The replication of this study on a larger 
sample was suggested. 
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Chapter 1 
INTRODUCTION 



Software complexity is a general, non-standard, and 
relative term describing the composition of the system, it 
is a relative term because it does not have an absolute 
value assigned to it. A system with high software 
complexity may be less complex than another system. The 
term is non-standard as it is not delimited in scope and 
may be used on different occasions to mean different 
things. A system with a large code having several 

interl iking modules may be considered a complex system. On 
the other hand a short program with a difficult algorithm 
may equally be called complex. 

Because of the generality associated with the term, 
software complexity may be used to define complexity level 
for various components of the system i.e. algorithm 
complexity, code complexity, programming language 
complexity, module linkage complexity, or I/O complexity. 
Software complexity may also be used as an index for 
various stages of system development and maintenance. 
These developmental phases may be system planning, system 
characteristics, system design, system tes ting, system 
management, system coding, and programming style. 

The term * software complexity'' gives an overall view 
of the complexity level in a system and does not 
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necessarily mean that all system components have the same 
complexity level. For example^ a system may have a complex 
code but a simple testing procedure or may have a complex 
design but simple cede. Likewise^ various stages of system 
development may not have the same complexity level. A 
system may be very simple at the design stage but very 
complex in the testing phase. Therefore, the term 
•software complexity' gives a general impression about the 
system but does not completely describe all its attributes. 

The complexity of a system is very much affected by 
the way the system is designed, coded, and tested. Human 
factors including programming style, language 
characteristics, and hardware limitations also affect the 
complexity of the system. It is a known fact that 
maintenance of a complex system is a difficult and 
expensive task. This concern has resulted in a research 
activity to find ways to develop less complex systems. The 
present study is a step forward in this direction. The 
identification of factors that affect softwc^re complexity 
will help computer scientists to build less complex 
systems • 

Statement of the Problem 

The problem addressed in this research study was 
stated as 'Identification of factors that affect software 
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complexity'. The term 'factors' as used in the statement, 
referred to the underlying dimensions that supposedly 
affect software complexity. The term software complexity 
was used interchangeably with system complexity and 
referred to large rjrograms only. Complexity was defined in 
terms of system maintenance and included repair 
maintenance, adaptive maintenance, and productivity 
maintenance. The definitions of these terms are given 
below: 

System Maintenance : 

System maintenance was defined as an amouont of effort 
needed to add, delete or modify segment (s) of a program. A 
program that takes less effort in modification is less 
complex than the one that takes more effort. 

Repair Maintenance : 

Repair maintenance is the maintenance needed to 
correct logic errors discovered in a program after it has 
been released into production (Vessey and Weber, 1983) . 

Adaptive Malntence i 

Adaptive maintenance is the maintenance needed by a 
program to better meet users' needs (Lientz and Swanson, 
1981) . 
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Productivity Maintenance ! 

Productivity maintenance is the maintenance needed to 
improve the efficiency of the program in terms of 
consumption of resources (Lientz^ Swanson^ and Tompkins, 
1978) . 

Significance of the Study 

Several attempts have been made in the past to 
identify the factors that affect software complexity and 
to develop a procedure to measure the complexity level in a 
system (Curtis^ Sheppard^ Millman^ Borst^ and Love^ 1979; 
Lientz et. al., 1981; Vessey et. al., 1983). 

The uniqueness of this study stems from rationale that 
overall system complexity cannot be measured accurately 
unless the complexity of its major components is measured 
correctly y because system components tend to have 
different levels of complexity. In order to produce an 
index for the overall complexity of a system, component 
complexity should be measured such that it has an additive 
property. The other unique characteristic of this study 
is the belief that overall system complexity is not of much 
value compared to the system component complexity. It is 
the component complexity that determines the cost of 
particular system maintenance. 
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The significance of this study therefore, lies in its 
attempt to identify major components of the system and the 
various elements that affect the complexity of those 
components . 



Purpose of the Study 

The purpose of this study was to develop an exhaustive 
list of el#>'^ents that affect system complexity measured in 
terms of system maintenace and then to use these elements 
to identify underlying dimensions called factors, that 
affect system complexity of various system components. 
The factors identified based on statistical significance 
were considered to provide a framework to conduct 
experimental study to determine the impact of each of these 
factors on various softwares under various conditions and 
to help in developing procedures to make less complex 
systems . 
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Chapter 2 
REVIEW OF LITERATURE 

Software complexity may have different meanings in 
different contexts. It may refer to the complexity of the 
algorithm, complexity of developing the system, complexity 
of interlinking modules, or the complexity of I/O 
interface. In the present study, software complexity means 
the amount of the effort needed to do repair maintenance. 

Software complexity is affectted by several factors. 
Structured programming, module programming, and top down 
programming have considerably reduced the software 
complexity of systems (Al-Suwaiyel, 1983). These factors 
contribute to the simplicity and clarity of code and 
therefore make it easy to perform repair maintenance 
(Canning, 1972). Formal proofs of correctness also reduce 
software complexity (DeMillo, Lipton, and Perlis, 1979). 
Personnel characteristics of programmers affect system 
complexity as well (Weinberg, 1971). Endres (1975) found 
empirical evidence that high quality programmers have less 
cost of repair maintenance. However, more formal and 
operational measures of module complexity, progrr-^ing 
style, and programmer quality are needed to reduce system 
complexity (Vassey et.al. , 1983). 

The issue of software complexity is tied up with the 
repair maintenance of the system. Vassey et. al., (1983) 
considered repair maintenance a function of system 
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complexity. Repair maintenance implies modifications in 
the program already in production to (1) fix logical errors 
discovered after its release, (2) to improve the 
operational efficiency of the program by economizing 
resources, or (3) to meet users' needs in a better way 
(Lientz et. al. , 1981) . 

The complexity measure provides an index of relative 
cost to implement or comprehend a system. Rising cost of 
software development and maintenance has aroused interest 
in tools and measures to quantify and analyse software 
complexity (Jensen and Nairavan, 1985). Leintz et. al. 
(1978) found that the repair maintenance cost may go from 
*.0% to 75% of the total life cycle of a system. Davis 
(1974) discovered the relationship between system 
complexity and the entropy. He found that complex systems 
undergo greater entropy. The high correlation between 
system complexity and repair maintenance as well as the 
need to reduce the cost of system maintenance has triggered 
tremendous amouont of research activity on issues related 
to system complexity (Thayer, Lipow, and Nelson, 1977) . 

The measureme.it of software complexity is still in its 
infancy (DeMillo, et. al., 1980). It is generally measured 
by the nuiube> of modules, size of modules, and inter- 
linkage of modules (Al-Suwaiyel, 1983). Software metrics 
are also used in decermining the system complexity. A 
strong relationship between software metrics and system 
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complexity has been identified by McCabe (1976) , Henry, 
Kafura, and Harris (1981), ottenstein (1981), and Schneider 
(1981). In spite of *-*^ese indices, it is very difficult to 
detenninp system complexity accurately because of less 
precise factors that determine such complexity. Such 
factors include but are not limited to ease of design, 
clarity of code, ease in understanding inter-module 
communication, programming style, and team work of 
progrzunmers (Pashtan, 1985) . 



Types and Levels of Complexity 



Al-Suwaiyel (1983) has categorized software complexity 
into inherent complexity and complexity of developing the 
system. Inherent complexity deals with the complexity of 
the algorithm and depends on the computational model used 
for the solution (Savage, 1976) . It may be defined as the 
minimum over all the complexities of all the algorithms 
available solution. Inherent complexity is generally 
measured in terms of time and space needed for execution. 
Quality and effectiveness of design and the total project 
cost can best be determined through inherent complexity. 

The second type of complexity that deals with system 
development have attributes like size of the system, number 
of modules, number of functions within the system and the 
linkage between modules. It is heavily influenced by 
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system design methods that include flow oriented methods, 
data structure oriented methods, and perspective methods 
(Peter, 1981) • System development complexity is 
independent of the inherent complexity of a system. 

Software comrlexity may also be categorized as (1) 
logical, (2) structural, or (3) psychological. Logical 
complexity deals with program control graph and may be 
measured by a graph model called Cyclomatic (McCabe, 1976). 
Structural complexity deals with size and linkages within 
the sytem. Psychological complexity, on the other hand, 
refers to program comprehensibility. 

The complexity level in a system may be documented in 
several ways. 7assey et. al., (1983) based his 
classification on the length of code. A system with a code 
length of 1 to 300 lines was called a simple system. Code 
length of 301 to 600 lines represented a moderately complex 
system while systems with more than 600 lines were labeled 
as complex systems. 



Software Metrics 

The software metrics that are most commonly used in 
determining software complexity have been reviewed briefly 
in the following text. 
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Program Length ; 

This is the most commonly used metric and is denoted 
by H such that 

N = + where 
and NNj represent total number of operator and operand 
occurances, respectively. This metric was first introduced 
by Halstead (1977) to measure software complexity. Program 
length can also be measured as 

N " n^ logj n^ + log^ where 
Ilj^ and jij represent unique operators and opernads in the 
system, respectively. 

If size of the system is the sole determinant of 
system complexity, program length is the most suited 
metric. Davis (1984) criticised the use of operator and 
operand counts and considered it as an obsolete measure to 
determine software complexity. Jensen (1982) however, has 
proposed anohter alternative to measure program length and 
it is represented by the following expression • 

N ^ log^ n^^ ! + log2 I 
This equation is considered an improvement over 
Halstead's original proposal (Jensen et. al. , 1985) . 
Jensen et. al. (1985) also reported that the normalized 
difference between the program length and Halstead »s length 
egtittAtor it higher than reported by Halstead (1977). it, 
therefore, confirmed the findings of Shen, Conte, and 
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Diinsxaore (1982) that program length is an imprecise measure 
to determine software complexity. 

Program Volume! 

Program Volume is the size of the program and is 
denoted by Y* Mathematically, 

V « (Nj^ + Nj) log^ {n^ + n2) where 
^nd are the number of operators and operands that 
occur in a system, respectively. The n^^ and jij refer to 
unique number of operators and operands, respectively 
(Halstead, 1977) . 

Programming Effort : 

The amount of programming effort needed to perform 
repair maintenance on a system is considered an index of 
the complexity of that system. It is denoted by £ and may 
be expressed as 

E = V (n^^ / n^) / 2) where 
n-^ / 2I2 / and represent number of unique operators, 
ntimber of unique operands, and the number of occurances of 
opemads, respectively. This metric was also proposed by 
Halstead (1977) . 

Programming effort may also be interpreted as decision 
to implement an algorithm. It is highly correlated with 
program length and program volume measures. Programming 
effort, therefore, is also algebrically equivalent to 
E = (n^^ V) (36 n^) where 
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V refers to program volvme. The notation q. , q and N 
have already been defined. 

Shen et, al. (1983) has developed a linear 
relationship between progranming time and progranuning 
effort. It is mathematically represented as 

E « / V* where V = N log^ 

In this equation, effort refers to elementary mental 
discriminations, V to the number of nental comparisons 
needed to write a program of length V* refers to 

program potential volume (smallest value) • 

Pashton (1985) has used programming effort metric to 
compare process model and monitor model in the design of 
operating systems. 

McCabe's Metric ! 

McCabe's metric denoted by MC deals with program 
control complexity. According to th:\3 scheme, lineraly 
independent control paths are determined in a connected 
graph representing the software system. Mathematically, 

MC «= e - v + 2 where 
e and y are the idges and vertices of the graph (McCabe, 
1976) • The metric MC may also be computed readily by the 
equation 

MC « D + 1 where 
jD denotes the number of decision nodes in the graph. 
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The correlation of program length, program volume, and 
programming effort with McCabe's complexity measure (MC) 
was found low and confirmed the hypothesis that high 
correlation between MS and H is typical of decision bound 
systems (Henry et. al., 1981). Prather (1984) considered 
Mccabe • s metric insensitive to restructuring of code , 
nesting level in the program^ and found it correltaed 
highly with the length of the program. 

Bond Metrics ! 

This metric was introduced by Belady (1980) and refers 
to the average level of nesting or average width of the 
control graph. It is also called Belady 's Bond Metric and 
is denoted by £. Mathematically^ 

B «= ( i L(i)) / K where 
K refers to the number of nodes in the control graph while 
L(i) denotes the number of nodes at level i. Jensen et. 
al., (1985) found consistently high correlation between 
McCabe's JJC and Belady •s S measures. However, because of 
less precision of B over MC and because of the 
computational ease, MS was considered a better estimate of 
system complexity. 

Program chunk t 

Davis (1984) has introduced a new metric called 
program chunk to measure software complexity. The 
rationale of this appraoch is that experienced programmers 
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use chunks to understand a program while beginners 
concentrate on individual statements (Shneiderman, 1976b). 
Therefore, the cost of repair maintenance can best be 
estimated if system complexity is expressed in terms of 
chunk complexity. 

Based on the work of Woodfield (1980) on modules and 
their inter-relationship, Davis (1984) proposed the 
following equation to determine chunk complexity. 

C - " ( ™ R^) where 
C = Complexity of the chunk i, 

m =« Number of other chunks affected by the chunk i, 
n « Number of chunks, and 
B " 2/3, a constant. 

The constant B is based on the assumption that the 
chunk is reviewed as many times as there is inter- 
relationship with other chunks and that the complexity of 
the chunk in terms of understanding it, decreases with 
every subsec[uent review. 

From the perspective of the maintenance programmer, 
the effort required to modify the program depends on 
whether the code is familiar or unfamiliar. The idea of 
chunks, 'jherefore, may easily be extended to familiar and 
non-familiar chunk. A chunk is considered familiar if it 
hikm ft high frftquftnoy of occurance. The formation of chunks 
recognizable by a programmer is a function of programming 
language and the application environment. It is because 
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semantic representation is formed by the syntactic 
knowledge of the programming language. For example, the 
recall of statements to switch the values of two variables 
is spontaneous to an experienced programmer. Moreover, the 
chunks that are meaningful can be remembered better 
(Shneiderman, 1976) . 

The identification of chunks and defining their 
boundaries is a difficult task. Norcio (1980) and Brotsky 
(1981) have addressed this issue in detail. 

Chunks may be classified as mandatory or non-mandatory 
(Mayer, 1979) . Mandatory chunks are a set of statements 
that have to occur together. For example, opening and 
closing of a FOR loop. Soloway (1982) however, has 
categorized chunks as strategic, tactical, and 
implementation . 

The overall complexity of a system, according to this 
metric, may be determined by the complexity of chunks and 
their inter-relationship. The latter is also called 
program structure. 

In spite of large number of research efforts, the 
software metrics that are available are not precise enough 
to accurately measure the system complexity. Moreover, the 
existing literature on the subject does not provide enough 
direction to rank these metrics in terms of their 
efficiency. The metrics of Halstead (1977) that are 
supported by Sanshara, Vehara, and Ohkawa (1981) and Curtis 
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et. al., (1979) are criticised by Baker and ^weben (1980) 
for being insensitive to program nesting and program 
structure. Similarly, the metrics of McCabe (1976) and 
Belady (1980) are considered too coarse to tap the 
intricasies of the progreun (Prather, 1984). The chunk 
complexity metric that taps the cognitive process needed to 
understand programs look promising but needs further 
research. 
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Chapter 3 
METHODOLOGY 

This chapter describes in detail the process used to 
identify factors that reduce software complexity measured 
in terms of repair maintenance. 

Development of the Questionnaire 

The first task of this study was to review the 
existing literature on software complexity and to identify 
elements that supposedly affect the complexity of the 
system* After a thorough review^ 160 items were selected 
as a potential set of elements that affect software 
complexity measured in terms of system maintenance • The 
items were generally drawn from Arthur (1983), Bersof, 
Handerson, and Siegel (1980), Dunn and Ullman (1982), 
Kernigham and Planger (1974), London (1974), Meek and Heath 
(1981), and Tassel (1978). 

Based on content, the items were categorized into nine 
logical categories named system planning, system 
characteristics, sty stem design, system testing, system 
documentation, system correctness, system clarity, 
programming style, and system management. System planning 
category included items related to system specifications 
and overall planning consideiations. System uxiaracteristics 



17 

23 



category included terms that defined various attributes of 
the system like efficiency, portability, maintainability, 
modularity, reliability and so on. Items related to design 
considerations were grouptad under system design category. 
System testing category included items on testing options 
and tools. Items that heavily influenced the correctness 
of the system output were grouped under system 
correctness. The items that influenced writing simple and 
easy to understand code were grouped under the program 
clarity category. Programming style elements describing 
Do's and Don'ts were grouped under programming style 
category. Items describing management of systems from 
planning to the completion stage were grouped under system 
management . 

Items were written like a Likert Scale to provide 
enough variance. Five choices numbered 1 to 5 were given 
to respondents on each item. The choices were labeled as 
strongly agree, agree, neither agree nor disagree, 
disagree, and litrongly disagree. For consistency 
considerations, the format of these choices was kept the 
same throughout the questionnaire. This forced some of the 
items to be worded negatively and they were spread randomly 
within the category to which they belonged. 

A statement describing the content of each category 
preceded before the items. In each lead paragraph, it was 
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explicitly mentioned that the system complexity should be 
interpreted in terms of system maintenance. 

After thn questionnaire was developed, a cover letter 
was prepared describing the purpose of the study, 
definition of the term 'complexity', and the use of the 
data collected. The need for respondents' cooperation was 
also emphasized. This cover letter was used as the front 
page of the questionnaire. 

The last section of the questionnaire requested 
demographic information on respondents. It included 
questions on sex, education, type of employment, nature of 
job, job designation, and computer related experience. 
This section was included to identify and exclude from 
study those respondents who were associated with software 
firms but were not necessarily computer scientists. For 
example, a Data Processing Manager who is an administrator 
working in a software firm may not have any formal training 
in the field of computer science. 

A copy of the questionnaire including cover letter and 
demographic section is given in Appendix A. 

Sampling 

Because of the length of the questionnaire, the 
cooperation of computer related businesses in administering 
the questionnaire to their employees was not frequently 
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available, since it was an exploratory study and intended 
to be exhaustive^ the reduced version of the questionnaire 
was not considered appropriate. Therefore, the whole 
questionnaire was mailed out or was delivered in person to 
businesses in Lawrence and Kansas City without any regard 
to sampling strategy. 

A total of 425 questionnaires were distributed of 
which 152 were received in complete form. The completed 
questionnaires were examined to identify the ones that were 
completed by computer professionals. This task was 
accomplished with the help of demographic information 
collected on respondents on the last page of the 
questionnaire. Seven of the completed questionnaires did 
not have complete demographic information to identify their 
status as computer scientists. These questionnaires were 
therefore excluded from the analysis leaving behind a 
sample of 147. 



Data Analysis 

The information from the completed questionnaires was 
entered on to a computer system. Item statistics in terms 
of means and standard deviation was computed for each item. 
Items with a mean value of 3.0 were considered non- 
discriminatory in terms of increasing or decreasing system 
complexity. Items with a standard deviation of 1.0 or 
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greater were either poor discriminators or the respondents 
did not have the theoretical or practical background of 
concepts used in those statements. 

Factor analysis was performed on each of the nine 
categories separately. A principal component solution was 
performed on each category to determine the total number of 
factors. The factors with eigenvalue of l.o or greater 
were considered significant. After this initial analysis 
for each category, factor analysis was repeated several 
times for each category by varying the number of factors to 
be extracted and examining rotated and unrotated solutions, 
until such a time th it a final solution was obtained. A 
solution was considered final if it was the most logical 
and meaningful in terms of naming factors and cross- 
loadings. Varimax rotation was used whenever a factor 
matrix was rotated. The assignment of items to factors 
depended on item loadings. The item was assigned to a 
factor on which it loaded the highest. Items with a loading 
of less than .40 were considered insignificant and were not 
assigned to any factor. Statistical Package for Social 
Sciences (SPSS, 1983) was used to perform statistical 
analysis. 
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Chapter 4 
RESULTS 



This chapter includes the results obtained from data 
analysis on items related to software complexity. The 
items, divided into nine categories called system planning, 
system characteristics , system design , system testing , 
system documentation , system correctness , system 
clarity, programming style, and system management, were 
analyzed within the respective categories. 

Table 1 describes the item means and standard 
deviations on all the items. The category to which items 
belong is identified on the top of each column. The 
analysis revealed that all the items on system planning had 
a mean value of less than 3.0. Item 15 was close to 3.0 
and was considered a non-discriminator of software 
complexity. Items 3 and 14 had a standard deviation 
greater than 1.0. 

Item statistics on items related to system 
characteristics revealed that items 2 and 14 have means 
greater than 3.0. This happened because these items were 
negatively worded. On the remaining items, the mean value 
was less than 3.0. In terms of standard deviation, items 1 
to 5, 8, and 14 had a standard deviation of greater than 
1.0. 
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Table 1 
ITEM STATISTICS 



System Planning 



ITEMS 


MFAM 


c n 


1 


X • ^ / 


n A c; 
U • 4 0 




1 AO 


0 • 70 




O TO 
^ • ±7 


1. 17 


A 


1 • 92 


0. 94 




2.12 


0.95 


it 
O 


1»58 


0.95 


7 


1.92 


0.98 


Q 

o 


2 . 00 


0.85 


y 


2 . 17 


0. 64 


10 


2.27 


0.83 


11 


2.42 


0.99 


12 


2.00 


0.89 


13 


2.04 


0.87 


14 


2.58 


1.03 


15 


2.92 


0.89 


16 


2.00 


0.85 


17 


2.12 


0.86 



System Characteristics 



ITEMS 


MEAN 


S.D. 


1 


2.19 


1.10 


2 


3 .50 


1.11 


3 


2.04 


1.15 


4 


2.46 


1.27 


3 


2.85 


1.16 


6 


1.46 


0.71 


7 


2.58 


0.99 


8 


1.36 


1.02 


9 


2.08 


0.85 


10 


1.77 


0.77 


11 


2.23 


0.91 


12 


1.5-1 


0.81 


13 


1.42 


0.58 


14 


3.42 


1.24 


15 


1.58 


0.76 
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TABLE 1 (Contd.) 
System Design system Testing 



ITEMS 




s,p. 


ITEMS 


MEAN 


S.D. 


1 


3.85 


1.05 


1 


1.85 


0.68 


2 


2.86 


1.21 


2 


2.39 


0.64 


3 


2.12 


0.71 


3 


2.04 


0.60 


4 


2.19 


0.75 


4 


2.08 


0.89 


5 


1.23 


0.43 


5 


2.00 


0.80 


6 


2.35 


0.69 


6 


2.00 


0.89 


7 


1.85 


0.78 


7 


2.12 


0.77 


8 


2.58 


0.86 


8 


1.73 


0.67 


9 


2.23 


0.71 


9 


2.39 


0.75 


10 


2.85 


0.83 


30 


2.04 


0.82 


11 


2.69 


1.23 


11 


2.58 


1.27 


12 


3.00 


0.80 


8 


1.73 


0.67 


9 


2.23 


0.71 


9 


2.39 


0.75 


10 


2.85 


0.83 


10 


2.04 


0.82 


11 


2.69 


1.23 


11 


2.58 


1.27 


12 


3.00 


0.80 


16 


2.48 


1.16 


17 


1.96 


1.08 


17 


2.04 


0.54 


18 


2.23 


0.86 


18 


1.80 


0.65 


19 


1.85 


0.88 


19 


2.40 


0.71 


20 


2.54 


0.91 








21 


3.58 


0.81 








22 


3.46 


1.14 








23 


3.73 


1.04 
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System Documentation 
ITEMS MEAN S.D. 



1 


3.80 


1.35 


2 


1.80 


0.76 


3 


1.80 


0.76 


4 


1.76 


0.78 


5 


1.68 


0.63 


6 


1.68 


0.56 


7 


1.52 


0.71 


8 


1.88 




9 


2-67 


0.82 


10 


1. 36 


0.65 


11 


1.84 


0.75 


12 


1.32 


0.56 


13 


2.20 


0.96 


14 


3.40 


0.96 


15 


3.24 


1.05 


16 


1.40 


0.58 



TABLE 1 (Contd.) 

System Correctness 



ITEMS 


ffEAN 


S.D. 


1 


2.24 


0.78 


2 


2.00 


0.82 








3 


1.77 


0.65 


4 


2.08 


0.80 


5 


4.04 


0.87 


6 


2.23 


0.95 


7 


2.19 


0.85 


8 


1.77 


0.71 


9 


1.60 


0.76 


10 


1.89 


0.77 


11 


1.92 


0.89 


12 


2.81 


0.90 


13 


3.40 


0.96 


14 


1.65 


0.75 


15 


1.88 


0.73 


16 


1.54 


0.81 


17 


2.27 


0.96 


18 


3.31 


0.88 


19 


2.19 


0.69 


20 


1.84 


0.75 
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System Clarity 
ITEMS MEAN S.D. 



1 


1.96 


0.72 


2 


4.19 


0.80 


3 


4.23 


0.91 


4 


1.85 


0.97 


5 


1.69 


0.55 


6 


2.00 


0.85 


7 


3.08 


1.09 


8 


2.00 


0.85 


9 


1.65 


0.49 


10 


2.04 


0.72 


11 


2.12 


1.03 


12 


2.65 


1.13 


13 


1.85 


0.54 


14 


1.73 


0.60 


15 


1.54 


0.76 


16 


2.12 


0.71 



TABLE 1 (Contd.) 

Programming Style 
ITEMS MEAN S.D . 



1 


2.00 


1.06 


2 


2.28 


0.98 


3 


1.89 


0.82 


4 


2.58 


1.14 


5 


3.62 


1.20 


6 


3.31 


0.93 


7 


2.46 


0.95 


8 


1.73 


0.67 


9 


1.69 


0.68 


10 


3.58 


0.90 


11 


2.27 


0.67 


12 


2.00 


1.04 


13 


2.62 


0.85 


14 


1.77 


0.59 


15 


1.73 


0.60 


16 


2.15 


0.78 


17 


2.04 


0.66 


18 


2.15 


0.78 


19 


3.81 


0.75 


20 


2.92 


0.80 
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TABLE 1 (Contd.) 

System Management 
ITEMS MEAN S.D. 



1 


2.23 


0.96 


2 


2.19 


0.90 


3 


1.69 


0.62 


4 


2.77 


1.03 


5 


2.77 


0.95 


6 


2.12 


0.86 


7 


2.58 


1.17 


8 


1.65 


0.69 


9 


2.19 


0.94 


10 


1.92 


0.63 


11 


2.35 


0.69 


12 


2.77 


1.03 


13 


2.15 


0.78 


14 


3.65 


1.23 
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The analysis of system design items revealed that 
items 1^ 16, and 21 to 23 had a mean value greater than 
3.0. Of these items, 1, 16, 21, and 23 were negatively 
worded and this kind of outcome was expected. Item 22, 
however, was found inappropriate to determine complexity 
because of its high mean value. The items, l, 2, 11, 17, 
22, and 23, had a standard deviation greater than 1.0 and 
reflected large variances present in the opinions of 
respondents. 

Item 15 was the only item on the list of items on 
system testing that had a mean value of greater than 3.0. 
It happened because of the negatively worded statement. It 
was also observed that the items 11, 14, and 19 had high 
standard deviations (S > 1.0). 

The analysis of system documentation items revealed 
that items 1, 14, and 15 were negatively worded and 
therefore, showed a high mean value (X > 3.0). Items 1 and 
15 were found to have a high standard deviation value 
(S > 1.0) . 

The analysis of items under system correctness 
category identified items 5, 13, and 18 with high 
mean values (X > 3.0). All these three items happened to 
be negatively worded statements . Regarding standard 
deviation, all items behaved as expected (S < 1.0). 

Items 2, 3, and 7 related to system clarity had high 
mean values because of their negatively worded stem. The 
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standard deviation of items 7, 11, and 12 was higher than 
1.0. 

The analysis of programming style items identified 
items 5, 6, 10, and 19 with high mean value (X > 3.0). 
Content of these items revealed that they were negatively 
worded. Standard deviation on items l, 4, 5, and 12 was 
found greater than 1.0. 

Item 14 related to system management had a high mean 
value (X > 3.0) because of its negatively worded statement. 
The remaining items of this category had a mean value of 
less than 3.0. Items 4, 7, and 12 were found to have 
standard deviation of greater than 1.0. 

Factor analysis was performed on items for each of the 
nine categories identified earlier. The results of the 
analysis are summarized below under the subheading 
representing each category. 

System Planning 

The principal component solution of items on system 
planning identified six significant factors. They together 
explained 77.4% of the total variance. The factors, their 
eigenvalues, and the percent of variance explained by each 
factor is given in Table 2. 

After exploring several possibilities, the solution 
with 3 factors was found meaningful. The original factor 
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TABLE 2 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM PLANNING ITEMS 



VARIABLE 



FACTOR 



EIGENVALUE PCT OF VAR CUM PCT 



PLAM 
PLAN2 
PLAN3 
PLAN4 
PLANS 
PLAN^ 
PLAN? 
PLANS 
PLAN9 



PLAN 
PLAN 
PLAN 
PLAN 
PLAN 
PLAN15 
PLAN- 
PLAN 



3,94697 
2« 94792 
2.14215 
1,5991 5 
1,35919 
1,16695 
.89620 
,69659 
.61 145 
.44935 
.35599 
.26114 
-21 328 
.15064 

•Mii 

.02378 



23,2 
17,3 
12.6 
9,4 
8.0 
6,9 
5,3 
4,1 
3,6 
2,6 
2.1 
1.5 
1.3 
,9 
6 
.4 
.1 



23,2 
40,6 
5^.2 
6.:, 6 
70,6 
77,4 
82,7 
86,8 
90,4 
93.0 
95,1 
96,7 
97,9 
98. S 
99,4 
99,9 
100.0 
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matrix was rotated orthognally to minimize crossloadings. 
The three factors explained 53.2% of the total variance. 
The factors, the items that loaded on these factors, and 
the loading of the individual items are given in Table 3. 

According to this solution, items 3, 4, 5, 10, 15, and 
16 loaded on Factor 1. Items 6 to 9 loaded on Factor 2. 
Items 1, 2, 11, 13, 14, and 17, loaded on Factor 3. Item 
12 loaded poorly on all the three factors and was therefore 
dropped from the analysis. Its highest loading was .38. 
Based on item content, the three factors were named as High 
level planning, design level planning, and Implementation 
level planning. 

The item loading on factor 1 ranged from .45 to .89. 
Item 16 had the lowest loading while item 5 had the 
highest loading. Item 4 also loaded high (.83). The 
remaining items loaded in .50's. On factor 2, items 6 to 8 
loaded in .70 's. The range of item loading was .63 to .77. 
Item 9 had the lowest loading. Item loadings on factor 3 
ranged from .50 to .70. The lowest loading was .50 on item 
11. Item 17 loaded the highest. 

System Characteristics 

The factor analysis of items on system characteristics 
found four significant factors that met Kaiser's criterion 
of eigenvalue greater than 1.0 (Kaiser, 1960). However, 
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Table 3 



Factors, items Loaded on Factors, and item Loadings 
on System Planning Category 



FACTOR 1 
High Level 
Planning 


FACTOR 2 
Design Level 
Planning 


FACTOR 3 

Impl ement at ion 

Level Planning 


icems 


Loading 


Items 


Loading 


Items 


Loading 


3 


.58132 


6 


.71252 


1 


.65085 


4 


.83292 


7 


.76632 


2 


.67906 


5 


.88704 


8 


.75154 


11 


.50273 


10 


.50264 


9 


.63042 


13 


.57553 


15 


.56487 






14 


.55251 


16 


.44937 






17 


.70456 
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three factor solution was found more appropriate and 
meaningful. The three factors combined^ explained 63.1% of 
the total variance. The fourth factor contiibuted 8.7% to 
the total variance. The factors, their eigenvalues, and 
the percent of variance as extracted by principal component 
solution are given in Table 4. 

Th€i analysis of the rotated factor matrix revealed 
that items 6, 8, 9, 12, and 13, loaded on Factor 1; items 
2f 3, 4, 5, 7, and 14, loaded on factor 2; and items 1, 10, 
11, and 15 loaded on Factor 3. The items that loaded on 
Factor 1 were related to system characteristics that dealt 
with maintenance activity. The factor was therefore named 
as Maintenance. Factor 3 included items that related to 
the correctness of the output and was named Correctness. 
The second factor included characteristics other than those 
related to Correctness or Maintenance, and was therefore 
named Others. The factors, the items comprising those 
factors, and their loadings are given in Table 5. 

Item loadings on factor 1 range from .64 to .86. The 
lowest loading was on item 12. Itme 8 loaded the highest. 
The range of item loading on factor 2 was .53 to .78. Item 
2 loaded the highest while Item 7 had the lowest loading. 
On factor 3, items 1 and 10 loaded in .50*s while items 
11 and 15 loaded in .80 *s. The range of lo;!fing was .57 to 
.898. Items 10 and 15 had the lowest and the highest 
1 oad ings , re spec t i ve ly . 




TABLE 4 

FACTORS. EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM CHARACTERISTICS ITEMS 



VARIABLE 


FACTOR 


CHAR1 


1 


CHAR2 


2 


CHAR3 


3 


CHAR4 


4 


CHARS 


5 


CHAR6 


6 


CHAR? 


7 


CHAR8 


8 


CHAR9 


9 


CHAR10 


10 


CHAR11 


11 


CHAR12 


12 


CHAR13 


13 


CHARU 


U 


CHAR15 


15 



EIGENVALUE 

3.75633 
3,71 272 
1.990A6 
1,31103 
-98471 
,71 253 
.64425 
.55377 
.37971 
.27914 
.20451 
.18207 
.13432 
.10076 
.05369 



PCT OF VAR 

25.0 
24.8 
13.3 

8.7 

6.6 

4.8 

4.3 

3.7 

2.5 

1.9 

1.4 

1.2 
.9 
.7 
.4 



CUM PCT 

25.0 
49. a 
63,1 
71 .g 
78.4 
83.1 
87.4 
91.1 

95.5 
96.9 
98.1 
99.0 
99.6 
100.0 
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Table 5 



Factors, Items Loaded on Factors, and Item Loadings 
on system Characteristics Category 



FACTOR 1 
Simplicity 


FACTOR 2 
Others 




FACTOR 3 
Correctness 


Items 


Loading 


Items 


Loading 


Items 


Loading 


6 


.78225 


2 


.77855 


1 


.59229 


8 


.85695 


3 


.68902 


10 


.56969 


9 


.72422 


4 


.70783 


11 


.83310 


12 


.64100 


5 


.76050 


15 


.89831 


13 


.76106 


7 
14 


.52989 
.64722 
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System Design 

The principal component analysis of items on system 
design extracted nine factors with eigenvalue of l.o or 
greater. They explained 35.1% of the total variance. 
After repeating analysis with different number of factors, 
the solution wiv.i five factors was found more meaningful. 
The five factor solution explained 60.9% of the total 
variance. The factors, their eigenvalues, and the 
proportion of variance explained by each factor are given 
in Table 6. 

The item loadings for the five factor solution after 
varimax rotation are given in Table 7. Factor 1, labeled a 
General factor, was comprised of items 2, 10, 14, 21, and 
23. The items 8, 17, 18, and 19, loaded on Factor 2 and 
this factor was named System Environment. Factor 3 was 
comprised of items 3 to 6 and 20. It was given a name of 
Programming Considerations. The fourth factor was called 
Programming Style and included items 1, 9, and 11. Items 
12, 13, 15, and 16 loaded on Factor 5 called Efficiency 
Considerations . 

Item 7 did not meet the criterion of minimum loading 
of .4 and was therefore dropped from further consideration. 
Item 22 was found inappropriate after the questionnaire was 
printed. Therefore, it was not included in any analysis. 
These deletions dropped the total number of items in this 
section to 22. 
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TABLE 6 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM DESIGN ITEMS 



VARIABLE 

D£S IGN1 

DESIGMI 

DESIGNS 

DESIGN4 

DESIGNS 

DESIGNS 

DESIGN? 

DESIGNS 

DESIGN? 

DESIGMO 

DESIGN11 

DESIGN12 

DESIGN13 

DESI6NU 

DESIGN15 

DESIGN16 

C1SIGN17 

DESIGN18 

DESIGN19 

0ESI6N20 

DESIGN21 

DESIGN22 

0ESIGN23 



FACTOR EIGENVALUE PCT OF VAR CUM PCT 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
1 
1 
20 
21 
22 
23 



3.61 591 
3.43662 
2,89156 
2. 10637 
U95081 
U65630 
1,59642 
1.25301 
1.07044 
•77270 
•67533 

• 51 980 
•40201 
.27990 
•20422 
•17545 
•11714 
•10450 
.08481 
.04894 

• 03216 
.00454 
.00106 



15.7 
14.9 
12.6 
9.2 
f..5 
7.2 
6.9 
5.4 
4.7 
3.4 
2.9 
2.3 
1.7 
1.2 
.9 
.8 
.5 
.5 
.4 
.2 
.1 
.0 
.0 



15.7 
30.7 
43.2 
52.4 
60.9 
68^ 1 
75.0 
80. 5 
85^ 1 
88.5 
91.4 
93.7 
95.4 
96.6 
97.5 
98.3 

98. 8 
99.3 
99.6 

99. 8 
100^0 
100.0 
100.0 
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Table 7 



Factors, Items Loaded on Factors, and Item Loadings 
on System Design Category 



FACTOR 1 

General 

Factor 


FACTOR 2 

System 

Environment 


FACTOR 3 
Program 
Consider- 
ations 


FACTOR 4 

Programming 

Style 


FACTOR 5 
Efficiency 
Consider- 
ations 


Items Loading 


Items Loading 


Items 


Loading 


Items Loading items Loading 


1 .74770 


8 .60084 


3 


.59183 


1 .73281 


12 


.64593 


10 .61624 


17 .83495 


4 


.65513 


9 -.60385 


13 


.66006 


14 -.60566 


18 .78685 


5 


.75944 


11 .71115 


15 


.74960 


21 .69443 


19 .74569 


6 


.66780 




16 


.62822 


22 .71095 




20 


.48605 








23 .58179 
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Item loadings on factor l ranged from .60 to .75. The 
lowest loading was on item 14 which also happened to be the 
only negative value on this factor. The loading on item 2 
was the highest. The highest loading on factor 2 was .83 
(item 17). The lowest loading was on item 8 (.60). The 
range of item loading on factor 3 was from .49 to .76. 
Item 20 had the lowest loading while item 5 loaded the 
highest, item 9 on factor 4 was the only negatively loaded 
item. It also had the lowest loading on this factor. The 
highest loading (.73) was found for item 1. The range of 
item loadings on factor 5 was .63 to .75. Item 15 loaded 
the highest while item 16 loaded the lowest. Items 12 and 
13 loaded .64 and .66 respectively. 



system Testing 

Seven factors having eigenvalue of 1.0 or greater were 
extracted from the 19 items on Testing. They together 
accounted for 75.3% of the total variance. The factors, 
their eigenvalues, and the proportion of variance explained 
by each factor is given in Table 8. 

After repeating the analysis with different number of 
factors, the solution with four factors was found to bb the 
best. The solution explained 56.5% of the total variance. 
Varimax rotation could not be performed due to 
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TABLE 8 

FACTORS, EIGENVALUES, AI'D THE VARIANCE EXPLAINED 
ON SYSTEM TESTING 
ITEMS 



VARIABLE 

T€ST1 

TES72 

TEST3 

TEST4 

TESTS 

TESTf 

TEST? 

TESTS 

TEST9 
TEST10 

TEST11 
TEST12 
TESTIS 
TESTU 
TESTIS 
TEST16 
TEST17 
TESTIS 
TEST19 



FACTOR 


EIGENVALUE 


PCT OF VAR 


CUM PCT 


1 


4.11 176 


21.6 


21.6 


2 


2.67095 


14.1 


35.7 


1 


2.194S1 




47.2 




1.76227 


n-.i 


56.5 


5 


1.27842 


6.7 


63.3 


6 


1.19170 


6.3 


69.5 


7 


1.10598 


5.8 


75.3 


1 


.99465 


5.2 


80.6 


10 


.85524 


4.5 


85.1 


.76674 


4.0 


89.1 


11 


.5851 2 


3.1 


92.2 


12 


,41027 


2.2 


94.4 


13 


.31239 


1.6 


96.0 


11 


.25821 




97.4 




.21 203 




98.5 


16 


.12395 


.7 


99.1 


17 


.08774 


.5 


99.6 


18 


.05368 


.3 


99.9 


19 


.02437 


.1 


100. c 
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nonconvergence. The factors and the items that loaded on 
these factors are given in Table 9, 

The items that loaded on Factor 1 included 1, 2, 3, 5, 
8, 13, 15, and 18, Based on item contents, the factor was 
named Test Planning. The second factor was named Testing 
Techniques and was comprised of items 9, 11, 12, 14, 17, 
and 19. Items 6, 10, and 16, loaded on Factor 3. The 
item content suggested Testing as an appropriate name for 
this factor. Factor 4 was comprised of items 4 and 7 only. 
This factor stayed in every solution examined and basically 
loaded all the items that could not be loaded on other 
factors. Therefore, this factor was given the name 
Residual. 

The item loadings on Footer 1 ranged from .50 to .76. 
Item 3 had the highest loading while item 13 had the lowest 
loading, item 15 was the only item with negative loading 
(-.66). Items 9, 11, and 19 loaded negatively on Factor 
2. Disregarding the sign of loadings, the range was found 
to be .50 to .75. The lowest loading was on item 17. Item 
12 loaded the highest. Items 10 and 16 on Factor 3 had 
almost the same loadings; .495 and .496, respectively. 
Item 6 with a loading of .58 was the only negatively loaded 
item on Factor 3. On Factor 4, item 4 loaded .63. The 
only other item on this factor was item 4 and had a loading 
of .63 
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Table 9 



Factors, Items Loaded on Factors, and Item Loadings 
on System Testing Category 



FACTOR 1 
Test 

Planning 


Testing 




Testing 




Residual 


Items 


Loading 


Items 


Loading 


Items 


Loading 


Items Loading 


1 


.65402 


9 


.61911 


6 


-.58582 


4 .63362 


2 


.69137 


11 


.54400 


10 


.49491 


7 -.58325 


3 


.75801 


12 


.75380 


16 


.49619 




5 


.56680 


14 


.56077 








8 


.51267 


17 


.49590 








13 


.49862 


19 


.52302 








15 


-.66493 












18 


.57430 













48 



system Docximentation 

P'-incipal component analysis of the 16 items assessing 
program documentation extracted five factors that met 
Kaiser's criterion of an eigenvalue (Kaiser, i960) and 
explained 72.7% of the total variance. The factors 
extracted, their eigenvalues, and the proportion of 
variance explained by each factor is given in Table lO. 

Factor analysis was repeated several times with 
varying number of factors until a three factor solution was 
found. The three factor solution explained 59.5% of the 
total variance. Varimax rotation provided a better look on 
factor matrix. The first factor was defined by items 2 to 
8, 12, and 16, was named as Documentation standards. The 
second factor named Quality and Style included items 10, 
11, 13, and 15. Items 1, 9, and 2 4 loaded on Factor 3. 
This factor was named Documentation Analysis. The factors 
and the item loadings on these factors are given in Table 
11. 

The range of item loadings on Factor 1 was .57 to .86. 
Item 3 loaded the lowest while item 7 had the highest 
loading. Items 6 and 8 had factor loadings in the .60's. 
The remaining items loaded in .70's. The range of item 
loadings on Factor 2 was .63 to .78. item 11 had the 
lowest loading, item 15 had the highest loading. The 
highest loading on Factor 3 was found on item 1 (.71). The 
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TABLE 10 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM DOCUMENTATION 
ITEMS 



VARIABLE 


FACTOR 


oolcui 


1 


D0CU2 


2 


0OCU3 


3 


DOCUA 


4 


0OCL5 


5 


D0CU6 


6 


0OCU7 


7 


D0CU8 


8 


0OCU9 


9 


DOCHO 


10 


0OCU11 


11 


oocuii 




0OCU13 


1§ 


DOCUIA 


U 


00CU15 


15 


D0CU16 


16 



EIGENVALUE 

5,3221 1 
2,49793 
1,69724 
1.06802 
1.04301 
.99134 
.88653 
,63145 
.46436 
.43418 
.34424 

.11284 
.08684 
.03530 



PCT OF VAR CUM PCT 



33.3 
15.6 
10.6 
6.7 
6.5 
6.2 
5.5 
3.9 
2.9 
2.7 
2.2 
1.4 
1.0 
.7 
.5 
.2 



33.3 
48.9 
59.5 
66.2 
72.7 
78.9 
84.4 
88.4 
91.3 
94,0 
96,1 
97,5 
98.5 
99,2 
99.8 
100.0 
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Table 11 



Factors, Items Loaded on Factors., and Item Loadings 
on System Documentation Category 

FACTOR 1 FACTOR 2 FACTOR 3 

Docxunentation Quality Documentation 

Standards and Style Analysis 

Items Loading Items Loading Items Loading 

2 .75081 10 .74884 1 .71028 

3 .57085 11 .63104 9 .62792 

4 .75785 13 .74112 14 .48123 

5 .78442 15 .77805 

6 .68614 

7 .85776 

8 .63960 
12 .74420 
16 .72493 
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lowest loading of .48 was on item 14. item 9, the only 
remaining item on this factor, loaded .63. 

System Correctness 

Principal component analysis performed on 20 items 
related to program correctness produced seven significant 
factors that met Kaiser's criterion of eigenvalues (Kaiser, 
I960). These factors explained 80.1% of the total 
variance. The factors, their eigenvalues, and the percent 
of variance explained by each factor is given in Table 12. 

The factor analysis was repeated several times by 
varying the number of factors to be extracted . The 
solution with four factors was finally selected for its 
logical and meaningful structure. This four factor solution 
explained 60.3% of the total variance. The factors and the 
items that loaded on these factors are given in Table 13. 
Item loadings are based on orthognally rotated solution. 

Items 7, 9, 10, 17, 19, and 20, loaded on Factor 1. 
Based on item contents, the factor was named as 
Programming Style. The second factor named Coding 
Considerations was comprised of xtems 5, 12, 13, 14, and 
16. Items 1, 3, 4, 6, and 15, constituted Factor 3 which 
was named Programming Structure. The fourth factor that 
included items 2, 8, and 18, was named Residual as the 
items that did not fit to other factors, were loaded on 
Factor 4. 



ERIC 



52 



TABLE 12 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM CORRECTNESS 
ITEMS 



VARIABLE FACTOR EIGENVALUE PCT OF VAR CUM PCT 



C0RECT1 

C0RECT2 

C0RECT3 

CORECT? 

C0RECT5 

C0RECT6 

C0RECT7 

C0RECT8 

C0RECT5 

C0RECT10 

C0RECT11 

C0RECT12 

C0RECT13 

C0RECT14 

C0RECT15 

C0RECT16 

C0RECT17 

C0RECT18 

C0RECT19 

CORECT20 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 



5.16206 
2.60216 
2.36562 
1 .92420 
1.54266 
1.37380 
1.04140 
• 96188 
.77504 
.62194 
. 5041 2 
•44440 
.27128 
.18606 
.06931 
.06382 
.04C67 
.03195 

•Mm 



25.8 
13.0 
11.8 

6.9 
5.2 
4.8 
3.9 
3.1 
2.5 
2.2 
1.4 

• 9 

• 3 

:i 

• 2 

:1 



25.8 
38.8 
50.6 
60.3 
68.0 
74.9 
80.1 
84.9 
88.7 
91.9 
94.4 
96.6 
98.0 
98.9 
99.2 
99.5 
99,8 
99, 9 

188:e 
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Table 13 



Factors, Items Loaded on Factors, and Item Loadings 
on System Correctness Category 



FACTOR 1 
Coding 

Considerations 


FACTOR 2 

Programming 

Style 


FACTOR 3 

Program 

Structure 


FACTOR 4 
Residual 


Items 


Loading 


Items 


Loading 


Items 


Loading 


Items Loading 


7 


.61130 


5 


-.66383 


1 


.87715 


2 .85286 


9 


.80668 


12 


-.41989 


3 


.67233 


8 .46509 


10 


.90569 


13 


-.49621 


4 


.79171 


18 .81536 


11 


.88483 


14 


.64655 


6 


.52551 




17 


.56146 


16 


.86023 


15 


.50372 




19 


.60276 












20 


.59991 
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Item loadings on Factor 1 ranged from .56 to .90. The 
items with the lowest and the highest loadings were item 17 
and item 10, respectively. Items 9 and 11 loaded in .80 's. 
The remaining items were loaded in .60's. items 5, 12, and 
13 on Factor 2 were loaded negatively. Disregarding the 
sign, the range of item loadings was .42 to .86. The 
lowest loading v^s on item 12 while item 16 had the highest 
loading. The range of item loadings on Factor 4 was .46 to 
.85. Items 2 and 8 were found as the highest and lowest 
loaded items respectively, item 18 loaded .81 on Factor 4. 

System Clarity 

Factor analysis was performed on 16 items on program 
clarity, six factors met Kaiser's criterion (Kaiser, 1960) 
and explained 79.4% of the variance. The factors, their 
respective eigenvalues and the percent of variance 
explained are given in Table 14. 

The factor analysis with three factors was accepted as 
a more logical and explainable solution. The initial 
factor matrix was rotated orthognally. The factors, the 
items that loaded on these factors, and their respective 
loadings are given in Table 15. 

Items 2 to 5, 11, 14, 15, and 16 were loaded on 
Factor 1. This factor was named as Program Quality. The 
s cond factor named Coding Considerations consisted of 
items 1, 6, 9, and 13. ' tems 7, 8, 10, and 12 constituted 
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TABLE 14 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM CLARITY 
ITEMS 



VARIABLE 

CLARI 

CLAR2 

CLAR3 

CLAR4 

CLAR5 

CLAR6 

CLAR7 

CLAfiS 

CLAR9 

CLAR10 

CLAR11 

CLAR12 

CLAR13 

CLAR14 

CLARIS 

CLAR16 



FACTOR EIGENVALUE PCT OF VAR CUM PCT 



3.92909 
2.78223 

hiim 

1.35292 
1.20630 

•Mm 

.52316 
.37365 
.27487 
.18721 
.16085 
.08972 
.06692 
.03645 



24.6 
17.4 

m 

8.5 
7.5 
5.5 
4.4 
3.3 
2.3 

]:l 

1.0 

.6 



24.6 
41.9 

ll:S 

71.9 
79.4 

1^:1 

92.6 
94.9 
96. 
97. . 
98.8 
99.4 
99.8 
100.0 
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Table 13 



Factors, Items Leaded on Factors, and Item Loadings 
on System Clarity Category 



FACTOR 1 

Programming 

Quality 


FACTOR 2 
Coding 

Considerations 


FACTOR 3 

Programming 

Style 


Items 


Loading 


Items 


Loading 


Items 


Loading 


2 


-.58995 


1 


.75874 


7 


.73963 


3 


-.78412 


6 


.81327 


8 


.56674 


4 


.54674 


9 


.51472 


10 


.66008 


5 


.63232 


13 


.52702 


12 


.68103 


11 


.62855 










14 


.56825 










15 


.74500 










16 


.44680 
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Factor 3, This factor was named Programming style based on 
the content of items defining the factor. 

Items 2 and 3 loaded negatively on Factor 1. The 
lowest loading was on item 16 (.45). Item 3 had the 
highest loading (.78). On Factor 2, the range of item 
loadings was .51 to .81. Items 6 and 9 were the highest 
and lowest loaded items, respectively. item 1 loaded .76 
while itme 13 loaded .53. The range of item loadings on 
Factor 3 was .56 to .74. Items 7 and 8 were the highest 
and lowest loaded items, renpectively. The remaining items 
were loaded in .60's. 



Programming style 

Tha principal component solution performed on 20 items 
of programming style extracted seven factors having 
eigenvalues greater than 1.0. The factors explained 80.2% 
of the total variance. Table 16 lists the factors, their 
eigenvalues and the percent of variance explained by these 
factors. 

After sevet«l trials, a four factor solution after 
varimax rotation was found more logical and explainable 
than the other solutions and was therefore adopted for this 
analysis. According to this solution, the four factors 
explained 60 . 0% of the total variance . Factor 1 was 
comprised of items 5 to 10, 12, and 19. The factor was 

Er|c 



TABLE 16 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON PROGRAMMING STYLE 
ITEMS 



VARI/)BL£ 

STYLE1 

STYLE2 

STYLE3 

STYLE4 

STYLES 

STYLE6 

STYLE? 

STYLES 

STYLE9 

STYLE10 

STYLEII 

STYLEli 

STYLE13 

STYLE14 

STYLE15 

STYLE16 

STYLE17 

STYLE18 

iTytiJ? 



FACTOR EIGENVALUE PCT OF VAR CUM POT 



1 
2 



5 
6 
7 
8 

9 
10 

n 

13 
14 
15 
16 
17 
18 



4,39826 
2.83706 
2.70445 
2.Q6803 
1.55852 
1.29267 
1.17177 
.89350 
.73479 
.58217 

.34431 
.3231 1 
.20276 
.18492 

.03994 
.01695 



22.0 

14.2 
13.5 
10.3 
7.8 
6.5 
5.9 
4.5 
3.7 
2.9 
2.6 
1.7 
1.6 
1,0 
.9 
• 6 
.2 
«1 



22.0 

36.2 

49.7 

60.0 

67.8 

74.3 

80.2 

84.6 

88.3 

91 ,2 

93.8 

95.5 

97.1 

98.1 

99.1 

99.6 

99.8 

99.9 

188:8 
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named Progam Quality. items 2, 2, 14, and 15, loaded on 
Factor 2. The factor was called Programming style. The 
third factor called coding Consideration included items 1, 
4, 11, 16, and 20. Items 13, 17, 18, produced the fourth 
factor called Efficiency Considerations. 

The factors, the items that loaded on these factors, 
and their respective loadings after varimax rotation are 
given in Table 17. 

Item 19 loaded slightly higher (.406) than the minimum 
value of .40, needed to assign that item to a particular 
factor. This item had the lowest loading on Factor 1. 
Item 10 had the highest loading (.87). Items 8, 9, and 12 
were negatively loaded. Item loadings on this factor were 
comparatively lower than the other three factors. Item 
loadings on Factor 2 ranged from .50 to .86. The items 
with the lowest and highest loading on this factor were 
items 15 and 2, respectively. The loading range on Factor 
3 was .61 to .85. Item 4 loaded the highest while item 11 
loaded the Icwest. Most of the remaining items had factor 
loadings in .60»s. The range of item loadings on 
Factor 4 was .75 to .88. Items 13 and 17 were the lowest 
and highest loaded items, repectively. The only remaining 
item (item 18) had a loading of .80. 
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Table 17 



Factors, Items Loaded on Factors., and Item Loadings 
on Programming style Category 



FACTOR 1 

Programming 

Quality 



FACTOR 2 

Programming 

Style 



FACTOR 3 
Coding 

Considerations 



FACTOR 4 

Efficiency 

Considerations 



1 



Itiii Loading Items Loading itiii Loading Items Loading 



5 
6 
7 
8 
9 
10 
12 
19 



.61753 
.58687 
.55423 
-.54315 
-.41708 
.87187 
-.48739 
.40658 



2 
3 
14 

15 



.86058 


1 


.70405 


13 


.74984 


.83146 


4 


.84666 


17 


.87682 


.68506 


11 


.60707 


18 


.80412 


.49848 


16 


.63895 








20 


.67019 
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system Management 

Fourteen management related items were analyzed by 
factor analysis to discover underlying dimensions. The 
principal component solution revealed six factors that 
explained 80.0% of the total variance. The factors, their 
respective eigenvalues and the percent of variance 
explained by each factor are given in Table 18. 

Factor analysis was repeated several times by varying 
the number of factors to be extracted so that a logical and 
meaningful solution could be obtained. The solution with 
four factors after varimax rotation met this criterion and 
is included here for interpretation. According to this 
solution, the factors explained 64.4% of the total 
variance. The factors, items loaded on these factors, and 
item loadings after varimax rotation are given in Table 19. 

Factor 1 was defined by items 2, 4, and 12, and was 
named Management Structure. Factor 2 included items 5, 8, 
9, and 10, and was called Personnel Management. Items 3, 
11, 13, and 14, comprised the third factor called 
Management Support. The fifth factor called Management 
Strategy consisted of items 1, 6, and 7. 

Item 2 loaded the lowest on Factor 1 (.59). The 
highest loading of .90 was found on item 12. Item 4, the 
only remaining item on this factor, loaded .65. The range 
of item loadings on Factor 2 was .60 to .81. The lowest 
loading was on item 9. Item 8 had the highest loading. 
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.iBLE 18 

FACTORS, EIGENVALUES, AND THE VARIANCE EXPLAINED 
ON SYSTEM MANAGEMENT ITEMS 



VARIABLE FACTOR EIGENVALUE PCT OF VAR CUM PCT 



MANAGE1 

MANAGE2 

MANAGE3 

MANAGE4 

MANAGES 

MANAGED 

MANAGE? 

MANAGES 

MANAGE9 

MANAGE10 

NANAGEII 

MANAGE13 
MANAGE14 



1 
2 

I 

5 
6 



9 

10 

]l 

13 

U 



3.64358 
1,96359 
1.88064 
1.53049 
1.12760 
1.05838 
.80897 
.62984 
.39490 
.32116 
.25376 
.21356 
.11252 
.06102 



26.0 
14.0 

10.9 
8.1 
7.6 
5.8 
4.5 
2.8 
2.3 



1 



.8 
-4 



26.0 
40.1 
53.5 
64.4 
72.5 
80.0 
85.8 
90.3 
93. 1 
95.4 
97.2 
98.8 
99.6 
100.0 
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Table 19 



Factors, Items Loaded on Factors, and Item Loadings 
on System Management Category 



FACTOR 1 

Management 

Structure 



FACTOR 2 

Personnel 

Management 



FACTOR 3 

Management 

Support 



FACTOR 4 

Management 

Strategy 



Items Loading Items 



Loading Items 



Loading Items Loading 



2 
4 

12 



,59563 
65539 
90512 



5 
8 
9 
10 



,65511 
,81564 
,59775 
64892 



3 
11 
13 
14 



.45150 
.52884 
.88239 
.75144 



1 .79499 

6 .74311 

7 .55144 



Si 



Item 5 was the only negatively loaded item on this factor. 
Item 14 on Factor 3 loaded negatively. The lowest and 
highest loading were on item 3 and item 13, respectively. 
The range of item loadings was .45 to .88. item loadings 
on Factor 4 ranged .55 to .79. Items 1 and 7 were the 
lowest and highest loaded items, respectively. item 6 
loaded .74 on this factor. 

The factor matrices on all the nine categories are 
given in Appendix B. 
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Chapter 5 
DISCUSSION AND CONCLUSION 

This chapter includes discussion of the results 
described in Chapter 4. The chapter closes with the 
conclusion derived from the findings of this research 
study* 

The analysis of item means and standard deviations 
confirmed that all the items included in the questionnaire 
affect system complexity to varying degrees. Items with a 
mean value of less than 3.0 reflected respondents' 
agreement that the attributes expressed in the statements 
decrease software complexity. Similarly, item means of 
greater than 3.0 reflected respondents' opinions that such 
attributes incre se system complexity. The items with a 
mean value of 3.0 or closer were not considered good 
discriminators. These items were PLAN 15, DESIGN 12, 
DESIGN 22 and STYLE 20. The content analysis revealed that 
all these items except DESIGN 22 were highly related to 
system complexity but respondents responc'ed to them 
randomly and the mean value turned out to be 3.0. 

An examination of standard deviation values revealed 
that not all respondents agreed or disagreed with the 
statements with equal strength. Standard deviation of 1.0 
or greater was viewed as high for this analysis. Standard 
deviation of less than 1.0 was an index of the conformity 
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of views of respondents regarding those statements. Large 
standard deviation values were considered to be a result of 
different interpretations, that respondents attached to 
standard terms. This fact confirmed the confusion that 
exists in computer science literature for non-standard 
definitions and use of more than one term for a single 
phenomenon . 

Factor analysis revealed that the nine logical 
categories to which the questionnaire was divided were not 
exclusive. Factor analysis of items of a single category, 
sometimes, produced factors that were related to the major 
category. For example. Factor 3 of system characteristics 
was named correctness which in fact is one of the nine 
categories. Similarly, Factor 4 of System Design, Factor 2 
of Program Correctness, and Factor 3 of Program Clarity 
discovered underlying dimensions that were supposedly 
tapped by one of the main categories. This finding 
confirmed the rationale of this study that a single factor 
in a system may affect several system components making a 
cumulative effect on the complexity of a total system. It 
meant that a system cannot be categorized into non- 
exclusive components. For example, system testing which is 
a component of system development may not stand alone 
because it includes planning for testing, characteristics 
of testing, testing documentation, testing management, 
testing accuracy, clarity of test procedures, and 
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programming style in test code. In other words, each 
component of a system encompasses the whole developmental 
span of the system. This finding was important for future 
research activity in which the factors obtained from this 
study may be used as logical entities rather than making 
judgmental categories as was done in this exploratory 
study . 

Factor analysis aljso revealed that a major category 
may give rise to factors that identify major components of 
that category. The example is of System Planning category 
in which all the three factors obtained refer to system 
planning but at a different level. Factor 1 referred to 
high level planning. Factor 2 clustered items that related 
to design level planning. The third factor identified 
implementation level planning or lower level planning. 
Similarly, the system testing category produced four 
factors called test planning, testing technique, testing, 
and residual factor. Another example was the system 
management category in which the four factors identified 
various components of management. Factor 1 was called 
management structure , Factor 2 related to personnel 
management. Factor 3 was management support, and Factor 4 
referred to management strategy. 

On program documentation category, the three factors 
identified three dimensions called documentation standards, 
quality and style of documentation, and documentation 



62 

68 



analysis. Under the programming style category, the four 
dimensions that were discovered by factor analysis were 
named programming quality, programming style, coding 
considerations, and efficiency considerations. 

On certain categories, factor analysis gave rise to 
factors that could not successfully be named. The examples 
included system characteristics category in which Factor 2 
was named "Others". It was, in fact, a residual factor 
such that the items that did not load on other factors were 
loaded on Factor 2. For the program correctness category. 
Factor 4 was a residual factor. The items that did not 
load on the coding consideration factor, programming style 
factor, or programming structure factor, loaded on Factor 
4 called residual factor. 

The cross-loading of items, though minimized by 
varimax rotation was still greater than expected. This 
confirms the earlier finding made on the basis of item 
means and standard deviations that the use of several terms 
for tho :,ame phenomenon means different interpretations. 
The other possible reason of cross-loadings was that 
people were not sure what they were responding to and 
therefore selected the degree of agreement or disagreement 
with the statement at random. This tendency might have 
been resulted from lack of education or experience by 
respondents in the respective areas. Most of the subjects 
of this study were involved in the process of large system 
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development but not all of them had extensive formal 
training in system design and development. Those who 
learned through experience were not familiar with the 
terminology used in the literature. The third category was 
those who had taken at least one course in system design 
and development but did not have experience of designing 
the system. The perceptions of these three distinct groups 
produced three unique sets of responses such that their 
cumulative effect resulted in cross-loading of items on 
more than one factor- 



Conclusion 

One hundred and sixty items extracted from the 
existing literature were categorized into nine categories 
according to their content. These categories were made on 
subjective reasons and were system planning, sys :em 
characteristics, system design, system testing, program 
documentation, program correctness, program clarity, 
programming style, and system management. 

Item means indicated that at least 98% of the items do 
affect system complexity. Content analysis indicated that 
all the items except one are related to system complexity. 
High variance in respondents * responses was attributed 
either to their lack of formal training or experience in 
some areas of system development, or to the confusion 
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caused by multiple definitions of a single term. Computer 
science literature has an abundance of terms that have 
multiple meanings and all of them are recognized as 
legitimate. 

Factor analysis on system planning category identified 
three factors called high level planning, design level 
planning and implementation level planning. The system 
characteristics category produced two factors called 
simplicity and correctness. The third factor was a general 
factor. Under system design category, the factors were 
identified as general factor, system enviornment, program 
considerations, programming style, and efficiency 
considerations. Test planning, testing technique, testing, 
and residual were the factors identified in System testing 
category. Program documentation category had three 
underlying dimensions called documentation standards, 
documentation quality & style, and documentation analysis. 
Program correctness had coding considerations, programming 
style, program structure, and residual as factors . 
Programming quality, coding considerations, and programming 
style were the dimensions of program clarity category. 
Programming style category identified programming quality, 
programming style, coding considerations and erficiency 
considerations as factors. The last category, system 
management, had management structure, personnel managexaent, 
management support, and management stragtegy as factors. 
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Factor analysis suggested that the items may be 
regrouped according to the factors extracted to determine 
complexity of system attributes, system components, or of 
various phases of system development. It was recommended 
that the study be replicated with a larger sample and that 
the complexity of various system components be weighted to 
determine the overall complexity of the system. 
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Th« Univorsity of Kansas 
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Institute for Research In Learmna OisabAties 

^mdYtiunQAtt^m 



Desr Conputer Scientists, 



There are several factors that are considered very predictive of softwar* 
I' stnictured^well docunen{ed72;d has lir^X'Z^ 
ut J^^Lflt ll ^li!^*^ ^ ^^''^^ maintenance after its release 

^ ^ programs that do not incorporate these factors! A 

l^ull^l l^^' -edifications if programs easy. ?s 

t H««ver. there is no empirical evidence as to 

fSln JfLI^.r^ ^? ^ ^^^"^^^ The present study is a step 
I^^mY^ identify the nost significant factors that reduce syston conplexity 
and to discover their underlying dinensions. conpiexiiy 

»hiw coiVlexity and software complexity are used interchang- 

?n ^i? ^r^^S^ questionnaire and both refer to large programs, only 
of .fflL^^Li^ ^r^i'l^ complexity is also delimited [Hn m^nt 
«a2ir^.^^J^.!!^?\^^*^* •"^^'y segment(s) of a progr«n. For 



tMtul^f IS^. 5 partcipstion in this study as a represen- 

JSJiiLr .SLSI^'*;*^"' develoixnent and miintenancS of 

Sr^lSah?^ fS^J?l!i: ^ questionnaire will provide 

us valuable information about factors that could reduce system complexity. 

I?"!^**^?!! ^.V^ questioiwalrt win be kept confidential and a 
^S2t be sent to you. if desired. If you have any question 

Thankyou for your participation and cospleting the questionnaire. 



Sincerely, 



Javaid Kaiser. Ph.D. 



Symttm plraalngCtop l^el dfttign) i« thm mo9t cmci.l p.rt of systea 
d«T«lopMt. Tboroufh underst«adU| of thm proposed s rstca not only 
helps Id its derelopMnt, but siso reduces systea complexity. Indicste 
the degree to which you sgree or dissgrce that the followiog cooeidere- 
tions St tile plsnniog stage would reduce the coaplexity of the systea. 
(Coeplczity, for this study, is defined in t^cas of repair aaintenaace) 



Circle 


1 


if you snumCLY AGREE with the statement. 




Circle 


2 


if you AGREE with the ststenent. 




Circle 


3 


if you ffEiniER AGREE OR DISAGREE with the stetea 


ent. 


Circle 


4 


if you DISAGREE with the statcseot. 




Circle 


5 


if you STROMGLY DISAGREE with the statescnt. 





System design specificstiona 

Softwsre requiresent specificstiona 

Perfonaace Specifications 

Product specificstioas 

Project rstiooale 

Top level dealgo review 

Hodule design review 

Sate baae deaign 

Integration teat plan 

Selectioa of teat procedurea 

Configuration aanagCMnt 

Docuaantation atandarda 

Quality control plan 



3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
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Tool •pecificstioas 

Vendor survey and survillaoce 

Knowledge about future derelopaient plsni 

TUture aointeoance activity 



12 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



The folloviof cli«r«cteriatica repreeent • eystea with less co^lejiity. 
Indicate the degree to which you sgree or dissgree th«t the nsned 
chsrscteristic would slso reduce systen ciwplezity. (Cocplezity is 
defined ss repsir nsintenance) 

Circle 1 if you STROMCLT AGREE with the statement. 

Circle 2 if you AGREE with the atatesent. 

Circle 3 if you neither AGREE or DISAGREE with the atatcacnt. 

Circle 4 if you DISARES with the atatesent. 

Circle 5 if you STRONGLY DISAGREE with the atateacnt. 



Correctneaa of output 
SfficiencydUnlaised proceaaing tlM) 
Flexibility to aake enhanceaenta 
IoUgrity(Bow well the aoftwaro and data 

are protected 
Interoperability (Interface with other ayatcM) 
Haintainability(Activlty to 7.ocat« or repair 

errors). 

Portability (Change in nachirie enwiroMent) 
Reliability(Dcgree to which a ayatca ia 



2 
2 
2 

2 
2 

2 
2 



required to perform ita functions). 
Uaability(Effort to learn, operate, and use 

the ayatem). 
Tea tability (Structured tea ting to insure 

correctneaa) 
Traceability (Machine operated oieaaureaent 

for correctneaa). 
Siaplicity(Iflple»entatioa of functiona in 

■oat underatandable way). 
Hodularitydndependent functions linked 

together). 
Conciaiondapleaent a function with 

■infwiB code). 
Structured programing (Uae of U*THEM-ELSS etc.) 



2 3 

2 3 

2 3 

2 3 

2 3 

2 3 

2 3 

2 3 



Several factora need to be considered at the detailed dealgn atage of 
ayatea development to reduce aoftware complexity. Indicate the degree to 
which you agree or diaagrec that the named factor would reduce the com- 
plexity of the ayatem. (Complexity la defined in Ucw of repair main- 
tenance) 

Circle 1 if you STRONGLY AGREE with the atatcmeni.. 
Circle 2 if you AGREE with the atatesent. 

Circle 3 if you KBITHER AGREE OR DISAGREE with the atatement. 

Circle 4 if you DISAGREE with the atatement. 

Circle 5 if you STRONGLY DISAGREE with the atateiMnt. 
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High decision deosityCl of decisions in s 
•odule). 

Bifh progrsa lereUl of CALLi to functions 
per 100 lines of code). 

Choice of procedures for forul corrective sction 

Kesolution of hsrdwsre snd softvsrs iaterfsce 

Hell defined dsts structure 

Bstsblishing control over bstches of input 

Use of Mthods for isolsting errors snd their csui 

Control on aultiprogrsaMing 

Dse of geaerslitf in progrsa design 

Etthsnceaent to s systea 

Fonul evslustion of algorittn sccurscy 

Differentisl coapsrison of progrsas 

Use of psudocode to docuaent aodule logic 

Use of grspbicsl tools to display softvsre logic 

Use of cross references in code 

Including aodules with aultiple entry/exists 

Choics of s progrsaaing Isngusge 

Adequacy of operstionsl envirooaenets 

Top-dovn progrsaaing 

State of the srt hsrdvsre for progrsa developaent 
Poor distinction between hardwsre sod software 

functions 
Initisl systca stste not considered 
Poor user trsining 



4 
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Testing is s psrt of systea developaent. The decisions asde during the 
testing stsge asy effect the systea coi»ple»ity. lodicste the degree to 
vhich yon sgrse or dissgrer that the oaaed chsrscteristic would reduce 
systea coapleaity. (Cosqilexity is defined ss repsir aaintenance) 

Circle 1 if you STRONGZ.T AGR££ with the ststeaent. 

Circle 2 if you AGREE vith the ststeaent. 

Circle 3 if you KEITUER AGREE OR DISAGREE with the ststeaent. 

Circle 4 if you DISAGREE with the ststeaent. 

Circle 5 if you SIKONGLT DISAGREE with the ststeaent. 



Choice of test procedures 
Choice of test equipaent selected 
Progrsa test snd opersting instructions 
Appropristeoess of tests of ressonahility for I/O 
vslidstion 

Estsblishing tolersnce for sccuracy criterion 

Top do«m testing 

Testing and aaintenance hiatory 

Quality of teat prograoning 

Run tiac analyaia 

Re tea ting of all aodulea on new data that interact 

vith aodified aodule 
Bottoa-up teating 

TcatiBf a big prog^aa in smII piecea 
Teating prograa at boundary valuea 
Checking anawcra by hand 



2 
2 
2 

2 
2 
2 
2 
2 
2 

2 
2 
2 
2 
2 



3 
3 
3 

3 
3 
3 
3 
3 
3 

3 
3 
3 
3 
3 



4 
4 
4 

4 
4 

4 
4 
4 
4 

4 
4 
4 

4 
4 



S 

s 

5 

S 
5 
5 
S 
5 
S 

5 
S 
5 
5 

5 
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Lack of exhaustive testiag 

UaiQg ■ sliaple versxoo to test the basic desiga 

UsiOf test d«ts for each path 

Adequate time for testiog 

Esch t«st represeatiDg s dlffereatiiig clsss 



2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 



Prograa docuoeotatioD i isportaat Id understaadiag the syttea logic, 
ladictte th'. degree to which you agree or disagree that the aaaed 
docuaeatatiOD characteristic would reduce systea vonplexity. (Coaplexity 
is dcfioed io tenia of repair DainteasDC^) 

Circle 1 if you STRONGLY AGREE with the statencat. 

Circle 2 if you AGREE with the ststcseat. 

Circle 3 if you NEITHER AGREE OR DISAGREE with the ststeaeat. 

Circle 4 if you DISAGREE irith the ststcacnt. 

Circle S if you STRONGLY DISAGREE with the ststeaeot. 

Inadequate dcacriptioo of data eoviroiiMeDt 1 2 3 4 S 
High self docuaeatatiOD value (# of coaaeat 

lines per 100 liaes of code). 12 3 4 5 

Docuoeatation for individual in tallation 12 3 4 5 

Developaent of operator aad aaintenance aaauals 12 3 4 5 
DocuaeDtstioD of iaput, output, and files handled 

by ths systea 1 2 3 4 5 

Information on specisl diagnostic codes and flags 12 3 4 5 

Docuaenting of coaplex logic, wiien used 1 2 3 4 5 



Docuaentation on interrupt processing 12 3 4 5 

Static analysis of docuacatstion snd source code 12 3 4 5 

Quality of written docuacnts 1 2 3 4 5 

Docuaentation of data Isyouts 12 3 4 5 

Agreeaent between coaaents and code 1 2 3 4 5 

UoiXoraity of style snd appearance 1^345 

Use o5 fsore coaaents than needed 1 2 3 4 5 
Indenting coaaents snd source code the 

saae aaount 1 2 3 4 5 

Docuaentstion should start at design stage 12 3 4 5 



Correctness is sn iJBportsnt characteristic of systea development. 
Several decisions that are made to make the systea function correctly 
slso affect its coaplexity. You are asked to iodicste the degree to 
vhich you agree or disagree that iapleaenting the naaed characteristic 
for correctness would also reduce softwsre coaplexity. (Coaplexity is 
interpreted as repair maintenance) 

Circle 1 if you STRONGLY AGREE with the statement. 

Circle 2 if you AGREE with the stateaent. 

Circle if you NEITHER AGREE OR DISAGREE with the ststeaeat. 

Circle 4 if you DISAGREE with the ststement. 

Circle 5 if you STRONGLY DISAGREE with the statement. 



Control structure te process priorities 
Conformity with dsts base rules 



3 4 
3 4 
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Cltrity io •ddreiiint ■cbeae 

Proper ute of regiiteri 

Pttchiog btd code ioiteid of revritiog it 

\}»9 of reoirtivc procedure! for recuriively 

defioed data ■tnictures 
Teniiiutinc input by cod-of-file or marker, 

Dot by count 
Identify bid output ind recover «#faen posiibls 
Inititlize variiblei ind constinti before uie 
Avoid off*by*on« error 
Brtnch the right wty on equality 
ArithtMtic with floating nuaberi 
Co^>triaon of flottlng point auaberi for equality 
Making code rlgbt before making it fitter 
Assure the correctneii of solution it the 

design stage 
Relisbility it iaportint than efficiency 
Inititlizing vtritblei with executable code 
Ute of aised dita typei 
Ute of debugging compiler 
Introducing debugging lidi eirly 



a 

2 
2 
2 
2 
2 
2 
2 

2 
2 
2 
2 
2 
2 



The level of clirity maintained during lyttem development, to a greiter 
extent, deteimiaei the eiie in maintenance, ifter the lyitem goet into 
production. Indicate the degree to vhich you agree or dlttgree Uut the 



following .tttement. about clirity would .l.o reduce lyatem complexity. 
(Complexity ii defined •■ repair maintenance) 

Circle I if you STRONGLY AGREE with the ■tatement. 

Circle 2 if you AGREE with the ■tatemcnt. 

Circle 3 if you HEITHER AGREE OR DISAGREE with the ststement. 

Circle 4 if you DISAGREE with the tUtemcnt. 

Circle 5 if you STRONGLY DISAGREE with the ■tatement. 



Sequence of lource code 

High GOTO deiuity(# of GOTO stitesenti per 100 
linei of code) 

Sicrifice cUrity for efficiency 

Trir.iform hard logical expression to limple ones 

Using mesningful itstement libels 

Use of uniform input format 

Using free*form input when possible 

Using blank spsces in source code 

Selecting Bnemooic nsmes that won't be confuaed 

Use of prefix or suffix on file names 
Using tingle sutcmeot per line 
Alphabstizing liats including irguments, 

psrameters, and declsrstions 
Use of pr»;,i4thcsis to svoid ambiguity 
IndenUlion to show program structure 
Making code simple to underatsnd 
Use of prefered vsrisbl« type for subscripts 



3 
S 
3 
3 
3 
3 
3 
3 
3 
3 

3 
3 
3 
3 
3 
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Progr»ini style elemeata eohaoce clarity md belp in understiod&og 
aoftvare logic better. Indicate tbe degree to wbich you agree or 
diaagree that the followiog aet of »tate«tnta would alao reduce aoftvare 
co«plexity. (Co^lexity ia interpreted aa repair aaioteiuDce) 

Circle 1 if you STROHGLY AGREE with the aUteaent. 

Circle 2 if you AGREE with tbe atatCMnt. 

Circle 3 if you NEITHER AGREE OR DISAGREE with the atateaent. 

Circle U if you DISAGREE with the atate^nt. 

Circle 5 if you STRONGLY DISAGREE with tbe atateacDt. 



Keeping aodule aize mil (not to excede 100 

executable atstenenta) 
0»e of tenporary variables 

Replacing repetetive taaka by CALLa to functiona 

Avoiding FORTRAN arithoatic IF 

Uae of uoneceaaary branchea 

Uae of conditional branches as a substitute 

for logical expreasion 
Uae of data arrays to avoid repetetive control 

aequeace 

Choice of data repreaentation that aakea the 

progran aiople 
MakiDg output aelf explanatory 
Strain to reuae code inatead of rearranging it 
Making apecial caaea truly apecial 
Don't diddle code to sake it faster, find 



a better algorithn 
Use of var&ablea not conatants for pararaetera 
Uae of library routinea and functxona when 

available 
Plan ahead for prograa chsnges 
Uae of coBpiler for auaple optiaixstion 
Block I/O efficiently 
Uae of load aodulea for repeated runs 
Uae of large number of NOT conditional clauses 
Uae of aeveral EJECT and SKIP stateaents 



Managenent playa an important role in aystca development and nay alao 
affect ayatea coaplexity. To what degree do you agree cr disagree that 
the following set of aanageaent related activltiea would reduce softwsre 
coaplexity. (Coaplexity ia defined aa repair aaintenance) 

Circle 1 if you STRONGLY AGREE with tbe stateaent. 

Circle 2 if you AGREE with tbe stateaent. 

Circle 3 if you NEITHER AGREE OR DISAGREE ^'ith the stateaent. 

Circle 4 if you DISAGREE vith the stateaent. 

Circle S if you STRONGLY DISAGREE with the atateaeat. 



Manageaent by objectives 

Phaaed methodology to develop aystea 

Favorable aanageaent envxronaenta 

Fixed «rh'Jule to cooplete work 



1 2 

1 2 

1 2 

1 2 



3 4 

3 4 

3 4 

3 4 
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Uottoger^out group of systea developer 

Tetm concept of tystea devtlopneot 

Egoless progrsaniag 

ProgrABaer's Motivatioa for task 

IncliwioD of deTclopaeot staff ia the testing teas 

Vertical and boritoatal iatersctioa of prograonera 

Veil budgeting of the systea 

A heirarchical organizatioo of progr«aaers 

S«£ll design teaas 

Differences over interpretation between 

project aaaager and general aaaagenent 



4 
4 
4 
4 
4 
4 
4 
4 
4 



'The questions below ask you responses to enable us to see the 
differences in opinions expressed by groups representing various levels 
of sex, education, experience, and profession. 

CIRCLE the response that describes you the best. 



SEX : 

EDUCATION: 



I. MALE 



2, FDW.E 



BS In computer science 

MS in computer science 

Ph.D. in computer science 

Computer coursework but no degree In Comp. Sc. 



5. No formal education In computer science 

TYPE OF EMPLOYMENT : 

1. Regular job 

2. Student monthly 

3. Student hourly 

NATURE OF JOB : 

1. Related to software development /maintenance 

2. NOT related to software development/maintenance 

3. Any other, explain; 



WORK DESIGNATION: 



COMPUTER RELATED EXPERIENCE : 

1. Less than 1 year 

2. 1-2 years 

3. 2-4 years 

4. More than 4 years 



Do you Kont o copy of re^&ull!'? 



1, Yes 



2. NO 
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FACTOR MATRICES 
System Planning 

ROTATED fACTOR MATRIX: 



PUAM 
PLA^2 
PLA^i 

PLA^4 

PLANS 

PLAK6 

PLA^? 

PLASa 

PLANS 

PLAMO 

PLAM1 

PLAKia 

PLAM3 

PLANU 

FLAM5 

PLAM6 

PLANl 7 



FACTOR 1 

-.15104 
.58132 
.65292 

-.2/#696 

-.^6897 
.39?09 
.U219 
.5U26^ 
.35332 
.U5230 

-.1/490 
.-51663 
.56487 
.44937 

-.U7418 



FACTOR 2 

.U33d3 
-.1 5660 

^.07077 

-.04583 
.71252 
.76632 
.75154 
.63042 
.44601 
• 47113 
.21101 
.53526 

-.08639 
.3 9333 
.03062 

-.0488/ 



FACTOR 3 

.65085 
.67906 
.2207U 
.03433 
.02189 

-.21493 
.18352 
. 1C46U 

-.11 709 
. l07bo 
.50273 
.38494 
.575b3 
.55251 
-.08636 
.38ft?y 
.70^56 



System Characteristics 

ROTATEC FACTOR MATRIX: 



CHAR1 

CHAR? 

CHARi 

CHAR4 

CMAP5 

CHAR6 

CHAfi7 

CHAR£ 

CHARV 

CHARIO 

CHAR11 

CHAH12 

CHAR13 

CHAR14 

CHAR15 



FACTOR 1 

-.00347 
-.^1096 
.33121 
.^6893 
-.05363 
.78225 
.^2812 
.85695 
.72422 
.448ft? 
.15897 
.04100 
.76106 
-.32250 
-.14548 



FACTO.? I 

.34353 
.77855 
.6t)902 
.70783 
•76050 
• 10112 
•52989 
.08343 
.04421 
-.34976 
-.06590 
-.44283 
-.04629 
.64722 
.18268 



FACTOR 3 

.59229 
. 10fc9y 
-.06656 
.0973/ 
• 12641 
.17049 
.527^3 
-.07460 
.1128^ 
. 56969 
.833lu 
.03491 
.02483 
.0941 1 
.89831 



ROTATED FACTOR MATRIX; 



System Design 



DESIGN! 

DESIGN2 

DESIGN3 

DESIGNA 

DESIGNS 

DES ICN6 

DESIGN? 

DESIGN8 

0eSlGN9 

DESJGNig 

DESlGNl 1 

DeSlGNl2 

DESIGNS 

0ESIGN14 

DESIGN15 

0ESTGN17 
DESIGN18 
05S1GN19 
DESlGNjn 
DES JCN2T 
DESUr422 
DESIGN23 



FACTOR 1 

•.43282 
.74770 
.2C638 
.13069 
-.03042 
-.28076 
-.31649 
.19998 
.17915 
.61624 
-.00776 
.32129 
-.01740 
-.60566 
-•13253 
-.10245 
.04388 
.03515 
-.07164 
-.J7246 
.09443 
.71095 
.58179 



FACTOR 2 

-.03483 
•08623 

-.41677 

-•47334 
.24867 
.06111 
.021 59 
•60084 

-.28404 
.1 6375 

-.01619 
.41374 

-mi 

.03710 
.1 742 5 
.83495 
•76685 
.74569 
.1 3822 
.05139 
.06567 
.099C4 



FACTOR 3 

-.01116 
.16588 
.59183 
.6551 3 
.75 944 
.66 708 
.19085 

-.20097 
.21633 
.21801 
-.1771 1 
-.14248 
.05232 
-.22110 
-.06722 
.07860 
.04731 
•48605 
-. 1 774<» 
-.23429 
.22461 



FACTOR 4 

.73281 
-.1 1820 
.021P7 
-.0693J 
-.14114 
-.10501 
-.21941 
-.23946 
.60385 
-.15111 
-.71 115 
-.04258 
.34787 
.0501 1 
.18171 
-.44542 

-.02580 
.22932 
-.02191 
-.10285 
.40169 
.47073 



1 1 J 

-.31?t.S 

•.L/V4 1 

. 1 74 4'V 
. AOVf /. 

-.(.43 W 
.1503 < 
.C45V 1 

-.^ 60Cft 
. 1 Uy 1 
.74000 
.628c7? 

- .1 5 •<o<' 
.01374 
. 1 0/V4 
.181 M 

- . 0 1 0 > 
.1 1200 
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BEST COPY AVAILABLE 



fACTCR MATRIX: 



System Testing 



TEST1 
TEST2 

nsi3 

TEST4 

TESTS 

TES76 

TES77 

JES18 

TES79 

TES710 

TEST11 

TEST12 

TEST13 

TESTU 

TESTIS 

TES716 

TES717 

TES718 

TES719 



FACrOH 1 

.6SA02 
.69:5/ 
.75801 
.37 366 
.S6.'30 
.AISiV. 

.S12/.7 

.01336 
.^9863 
.21568 
-.66493 
-•01SS3 
.27S32 
.S74iO 
.37301 



fACTOrf i 

.24662 
•1 S0S9 
.06604 
•.13508 
-.20941 
.05105 
-.41655 
-.27148 
-.6191 1 
-.02090 
-.54400 
•75380 
.31395 

rS6?26 

.09268 
.49590 
.28668 
-.52302 



FACTOR 3 

-.07002 
-.50114 
-.13370 

-.14248 
-.58582 
.1991 9 
.27751 
.42928 
.49491 
.30864 
.34504 
.36697 
.38014 
.40870 
.49619 
.18C6e 
.08620 
.02338 



FACTOR < 

.30fi8J 
. 0775:? 
.16354 
. 63362 
-.0729^ 
-.13921 
-.58325 
.02965 
•179SS 
-.07510 
-.?P4no 
-.27305 
. 21C33 
.26n4/# 
•41924 
-.43561 
-.11723 
38*'24 
.1V06V 



System Documentati 



on 



feOTATEO FACTOR WATRIX: 



OOCOl 

00C12 

D0C03 

0OCO4 

OOtOS 

00C06 

0OC17 

0OCO8 

DOCl'9 

OOCtIO 

00C111 

0OCO12 

0OC013 

OOCOU 

00CL15 

00C016 



FACTOR 1 

.16579 
.75081 
•57085 
• 75 785 
.78442 
.68614 
.85776 
.63930 
.U5428 
.28191 
.28680 
.74420 
•05643 
-.13553 
-.34241 
.72493 



FACTOR 2 

.U628S 
-.1621 7 
•00693 
.08008 
.30342 
.33187 
.06177 

• 16208 
.31812 

• 7/,884 
•65104 

-•01992 
•74112 
.36144 
./780S 

-.08370 



FACTOR : 

.7102d 
-.16901 
-.01 676 
.51574 
.00?41 
-. UV67 

-.62/92 
.22357 

-.18537 
.14 898 

-.0SC21 
.4B123 
.02 702 
.35595 



System Correctness 



ROTATED FACTO.? MATRIX: 



C0RECT1 

C0RECT2 

C0RECT3 

C0R£CT4 

CORECTS 

C0RECT6 

COKECl 7 

CORECTS 

C0RECT9 

CORECTIO 

C0RECT11 

C0RECT12 

CORECTIJ 

CORECTU 

C0RECT15 

C0RECT16 

C0RECT17 

C0RECT18 

COR6CT19 

CORECT20 



F/>CTOR 1 

\m\ 

.29996 
•24412 
.315/2 
•27935 
•O1130 
.41050 
.8066^) 
.90569 
.88483 
.18614 
-.00995 
.27380 
•01654 
.18763 
.56146 
.02717 
•60276 
•59991 



FACTOR 2 

-.20536 
-•01932 
-.36331 
.20626 
•66383 
••30270 
• 18329 
-.05453 
.03476 
.06539 
.00669 
-.41939 
-.4 9621 
.64655 
•36516 

-mw 

•06064 
-•00824 
^43457 



FACTOR 3 

.8771 5 
.30808 
.67233 
.79171 
.0661 8 
-.52551 
-.09592 
•05814 
-.00669 
.19019 
.17624 
.16192 
•14408 
•20442 
.50372 
.0906 5 
.1067/ 
-.22859 
-.11505 
• 1 4 64 8 



FACTOR 4 

.11700 
.85266 
-.19507 
.05936 
.2033!) 
-.06425 
-.C9?t47 
.46509 
.16437 
-.C5224 
.22241 
-.37423 
.1360a 
.18444 
-.28440 
-.09370 
-.03526 
.31536 
• 3407 ^ 
-.04995 
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System Clarity 



BEST COPY AVAILABLE 



ROTATED FACTOR f'ATRiX: 



CLARl 

CLAR2 

CLAR3 

CLAR4 

CLAR5 

CLAR6 

CLAR7 

CLARa 

CLAR9 

CLAR1C 

CLAR11 

CLAR12 

CLAR13 

CLARU 

CLAR15 

CLARU 



FACTOR 1 

.U5136 
.58995 

.63232 
.09616 

-.U512 
.06933 
.37322 

-.01660 
.62855 
.0:^884 

-.29952 
.56825 
.74500 
.44680 



MCTOk 2 

.75874 
-.io0l9 
.04369 
.iV463 
.50065 
.81327 
-.06990 
.54882 
.51472 
.34703 
-.00881 
-.01795 
.52702 
.04904 
.03383 
-.00544 



FACTOR 3 

. 36846 
-.11312 
-.1734V 
-.11504 
-.00005 
-.04049 
.73 96 3 
.56074 
-.05436 
.6600^ 
.56158 
.68103 
.03967 
.027V5 
.02994 
.2109i> 



ROTATED FACTOR MATRIX: 



Programming Style 



STYLEI 

STyLE2 

STVLE? 

STyL£4 

STYLES 

STYLE6 

STVL£7 

STYLE? 

STYLE9 

STYLE10 

STYLE11 

STYLE12 

STYLE13 

STYLEU 

STYLE15 

SrYLEl6 

STYLE17 

STYLE18 

STYLE19 

STYie20 



FACTOR 1 
•.3l'o87 



-•00102 
-•19090 
-.1A642 



.61753 
.58687 
• 5:>423 
.5-.315 
.41708 

:um 

•48739 
•4i621 
.20857 
.39010 
-.12152 
•UV3o6 
.05806 
-.40658 
-.12750 



FACTOR 2 

.31784 
•86058 
.83146 
•02383 
•24255 

-•01243 
•01458 
•34861 
.22865 

-•19134 

-.24933 
.3 780 5 
•25648 
•68506 
• 49848 
.1?378 

-•13670 
.2621 5 
•07875 
•38779 



fACTOR 3 
.70405 

-.I2<i94 
.84606 
.154fi6 
2V5 
(072 
.33483 
•0?43j 
.02304 
•60707 
.28041 

-.02834 
.235oO 

-.0002<» 
.63895 
.031^2 

-.04537 
.01V18 

-.67019 



FACTOR 4 

-.32558 
.01739 
.14365 
.C2205 
.16407 
.11146 

-.18305 

-•17823 
•27436 
.05527 
.22470 
.21431 
•74984 
.06009 
.09470 
. 52O70 
.«76K^ 
.80412 

-.21753 
.2264^ 



ROTATED FACTOR KATRIX: 



System Management 



HANACEl 

MANA(}E2 

MANAGE3 

HANAGE^ 

MANACc5 

^ANAGE6 

MANACE7 

P<ANACE8 

MANAGEV 

MANAGE10 

nUA£E11 

KAKA6E12 

MANAGE13 

HAKAGEU 



FACTOR 1 

-•15153 
.59563 
•17721 
.65539 
.31/33 
.20584 
•16346 
.0/265 
•32098 
•46326 
.49979 
•905^2 
•05233 

-.45037 



fACTOR 'i 

•28580 
•00052 
.44494 
•20701 

-.65511 
.09036 

-*35173 
•81564 
.59775 
.64892 
•25069 

-•05333 
.06770 

-.08803 



FACTOR 3 

-.26765 
.00504 
•45150 
.QJ117 
.32798 
•21199 
.071aO 
.06892 
. 13389 
• 27543 
•52884 

-.07958 
.88239 
. 75144 



FACTOR < 

•79499 
.23343 
.37460 
.01257 
.00050 
. 74311 

• 55U4 

• 1604 7 
-•18623 

.20455 

-Ioi^^6 

-.02109 
-.01843 
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